Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor project structure, enhance logic, update configurations, and improve code quality #85

Conversation

filipchristiansen
Copy link
Collaborator

@filipchristiansen filipchristiansen commented Dec 31, 2024

This PR refactors the project structure and enhances code quality by:

Refactoring and Logic Improvements

  • Refactoring the _scan_directory function in src/gitingest/ingest_from_query.py by extracting loop logic into the new _process_item function, and further separating functionality into _process_symlink and _process_file
  • Replacing multiple return statements with error raising and catching, introducing custom exceptions (MaxFilesReachedError, MaxFileSizeReachedError, AlreadyVisitedError) in the _process_item and _scan_directory functions
  • Enhancing the logic in the process_query function in src/process_query.py for better flow and maintainability
  • Improving the logic in _generate_token_string in src/gitingest/ingest_from_query.py
  • Refining the download_ingest function in src/routers/download.py for better clarity and functionality

Exception Handling Enhancements

  • Replacing broad Exception handling with specific OSError in the _read_file_content function in src/gitingest/ingest_from_query.py
  • Refining exception handling throughout the codebase, including removing redundant try-except-raise blocks, e.g., in clone_repo function in src/gitingest/clone.py
  • Adding custom exceptions to src/gitingest/exceptions.py: MaxFilesReachedError, MaxFileSizeReachedError, and AlreadyVisitedError
  • Including explicit re-raising of exceptions in various functions for improved error propagation

Test Suite Refactoring

  • Cleaning up and reorganizing test files:
    • Moving tests from src/gitingest/tests/ to tests/
    • Consolidating fixtures from tests/test_ingest.py into tests/conftest.py
    • Removing redundant content from tests/conftest.py
  • Migrating configuration from pytest.ini to pyproject.toml, deleting pytest.ini, and updated .dockerignore

Documentation Improvements

  • Adding darglint for enforcing numpy docstring style in .pre-commit-config.yaml for src/ files
  • Updating docstrings throughout the codebase, including adding module docstrings where needed
  • Updating README.md:
    • Adding "GitHub stars" badge
    • Moving the "Discord" badge to its own line
    • Replacing occurrences of "Gitingest" with "GitIngest" for consistency and clarity

Linting and Code Quality

  • Integrating pylint into .pre-commit-config.yaml for both src/ and tests/ directories
  • Creating tests/.pylintrc for linting configuration specific to test files

Code Clean-up

  • Removing the redundant src/__init__.py file

Naming Conventions and Code Style

  • Renaming logSliderToSize to log_slider_to_size in src/server_utils.py for consistency with Python's naming conventions
  • Adding explicit encoding specification in multiple instances of open throughout the code

@filipchristiansen filipchristiansen changed the title Refactor/logic structure and config v2 Refactor project structure, enhance logic, update configurations, and improve code quality Dec 31, 2024
@filipchristiansen filipchristiansen force-pushed the refactor/logic-structure-and-config_v2 branch 8 times, most recently from d867896 to 61dbc8b Compare December 31, 2024 10:26
Copy link
Owner

@cyclotruc cyclotruc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like somewhere in there a bug found its way:

After hosting the front-end i tried the gitingest example and got this buggy screen
image

It seems like the POST request returns wrong arguments

@cyclotruc cyclotruc added the changes requested A review requested changes before merging label Jan 2, 2025
… improve code quality

Refactoring and Logic Improvements

- Refactored the `_scan_directory` function in `src/gitingest/ingest_from_query.py` by extracting loop logic into the new `_process_item` function, and further separating functionality into `_process_symlink` and `_process_file`
- Replaced multiple return statements with error raising and catching, introducing custom exceptions (`MaxFilesReachedError`, `MaxFileSizeReachedError`, `AlreadyVisitedError`) in the `_process_item` and `_scan_directory` functions
- Enhanced the logic in the `process_query` function in `src/process_query.py` for better flow and maintainability
- Improved the logic in `_generate_token_string` in `src/gitingest/ingest_from_query.py`
- Refined the `download_ingest` function in `src/routers/download.py` for better clarity and functionality

Exception Handling Enhancements

- Replaced broad `Exception` handling with specific `OSError` in the `_read_file_content` function in `src/gitingest/ingest_from_query.py`
- Refined exception handling throughout the codebase, including removing redundant try-except-raise blocks, e.g., in `clone_repo` function in `src/gitingest/clone.py`
- Added custom exceptions to `src/gitingest/exceptions.py`: `MaxFilesReachedError`, `MaxFileSizeReachedError`, and `AlreadyVisitedError`
- Included explicit re-raising of exceptions in various functions for improved error propagation

Test Suite Refactoring

- Cleaned up and reorganized test files:
  - Moved tests from `src/gitingest/tests/` to `tests/`
  - Consolidated fixtures from `tests/test_ingest.py` into `tests/conftest.py`
  - Removed redundant content from `tests/conftest.py`
- Migrated configuration from `pytest.ini` to `pyproject.toml`, deleted `pytest.ini`, and updated `.dockerignore`

Documentation Improvements

- Added `darglint` for enforcing `numpy` docstring style in `.pre-commit-config.yaml` for `src/` files
- Updated docstrings throughout the codebase, including adding module docstrings where needed
- Updated `README.md`:
  - Added "GitHub stars" badge
  - Moved the "Discord" badge to its own line
  - Replaced occurrences of "Gitingest" with "GitIngest" for consistency and clarity

Linting and Code Quality

- Integrated `pylint` into `.pre-commit-config.yaml` for both `src/` and `tests/` directories
- Created `tests/.pylintrc` for linting configuration specific to test files

Code Clean-up

- Removed the redundant `src/__init__.py` file

Naming Conventions and Code Style

- Renamed `logSliderToSize` to `log_slider_to_size` in `src/server_utils.py` for consistency with Python's naming conventions
- Added explicit encoding specification in multiple instances of `open` throughout the code
@filipchristiansen filipchristiansen force-pushed the refactor/logic-structure-and-config_v2 branch from 77244e7 to 2c66967 Compare January 2, 2025 11:57
@filipchristiansen
Copy link
Collaborator Author

filipchristiansen commented Jan 2, 2025

Error looks to be resolved now. Tested locally. @cyclotruc

@cyclotruc cyclotruc merged commit d1b21d9 into cyclotruc:main Jan 3, 2025
7 of 8 checks passed
@filipchristiansen filipchristiansen removed the changes requested A review requested changes before merging label Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants